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ABSTRACT 


We  have  explored  the  literature  on  mental  workload  and  have  found 
serious  lack  of  consideration  given  to  the  intelligent  restructuring 
of  tasks  which  is  a  common  characteristic  of  skilled  operators  of 
complex  man-machine  systems.  Virtually  all  existing  methods  for 
measuring  or  estimating  the  workload  imposed  on  human  operators  by 
simulated  systems  involve  either  real-time  system  simulation  and  the 
use  of  real  human  operators,  or  the  use  of  data  banks  of  human 
performance  data.  Since  such  data  gathered  in  the  laboratory  are 
highly  synthetic  and  obtained  under  conditions  quite  unlike  the 
relative  freedom  of  choice  of  human  operators  using  real  systems, 
we  have  also  come  to  the  conclusion  after  much  analysis,  that  computer 
simulations  of  both  man  and  system  performing  a  well-specified 
mission  are  the  only  practical  way  to  attack  the  problems  of 
mission  delimitation,  machine  improvement,  and  operator  specification. 

These  almost  contradictory  statements  are  brought  into  consonance  by 
requiring  that  the  simulation  of  the  human  operator  possess  intelligence 
and  be  able  both  to  play  with  alternative  ways  of  doing  things  and 
(to  quote  Sheridan)  to  report  on  "[its]  private  subjective  experience 
of  [the]  cognitive  effort." 

The  work  on  mental  workload  that,  has  been  done  to  date  by  all  of 
those  in  the  field  is  largely  concerned  with  the  reductionist  approach. 

i  i  i 


All  have,  in  fact,  used  the  lowest  elements  of  behaviour,  for  which 


data  exist,  and  have  tacitly  assumed  that  the  linear  hypothesis  holds 
the  whole  is  no  more  than  the  sum  of  its  parts.  Proof  that  this  is 
true  for  intelligent  behaviour  is  not  likely  to  be  found  in  the  near 
future . 

We  thus  arrive  at  a  more  difficult  and  demanding  problem,  that  of 
artificial  intelligence.  Fortunately,  the  kind  of  intelligence  which 
must  be  simulated  is  not  completely  beyond  our  capabilities.  It  will 
probably  suffice  to  include  rational  decision  making  and  a  capacity 
to  solve  certain  classes  of  games. 
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Figure  2.  Diagram  showing  how  Time  and  Order  Constraints 


-  I  - 

INTRODUCTION 

With  the  continually  increasing  application  of  digital  hardware 
to  cockpit  instrumentation,  there  have  arisen  problems  relating  to 
the  methods  of  displaying  information  to  the  pilot  and  accepting 
inputs  from  him. 

As  the  pilot's  functions  become  more  and  more  those  of  a  systems 
supervisor  the  task  of  integrating  him  into  the  systems  design  becomes 
more  difficult.  In  particular,  the  continuous  control  loops  tend 
to  be  subsumed  by  the  computers  and  the  pilot  becomes  an  observer 
whose  principal  function  is  more  to  choose  amongst  various  alternatives 
than  to  execute  them. 

One  of  the  major  design  difficulties  stems  from  the  fact  that 
human  beings  appear  to  operate  best,  as  controllers,  from  spatially 
organized  information.  Such  information  is  inherently  difficult  to 
present  and  is  more  costly  in  terms  of  hardware  and  less  realizable, 
than  digital,  discrete,  symbolic  displays.  How  successful  the  pilot 
can  be  when  presented  with  digital  information  on  which  to  make  spatial 
decisions  is  a  matter  of  conjecture,  and  can  be  best  approached  through 
workload  estimation  procedures. 

One  direction  of  possible  solutions  lies  in  the  use  of  models 
of  human  behaviour  which  can  predict  satisfactorily  the  performance 
of  the  pilot.  Many  such  models  exist.  Among  these  are  the  quasi linear 
control  models  of  McRucr  et  al. ,  the  visual  sampling  models  of  Senders, 
and  the  taxonomic  structure  of  Teichncr.  This  latter  is  based  on  the 
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notion  of  the  single  channel  model  of  the  human  operator  and  is 
compatible  with  a  multi-dimensional  model  of  human  operator  workload. 
We  first  consider  these  models  and  their  application  to  the  solution 
of  the  problems  of  complex  digital  cockpit  design  and  man-machine 
integration . 


MENTAL  WORKLOAD 


General 


Mental  workload  is  a  concept  as  well  as  a  group  of  models.  The 
concept  is  created  by  analogy  with  the  concept  of  physiological  work. 

In  general,  it  seems  reasonable  to  assume  a  less-than-inf inite  capability 
to  do  "mental  acts."  Given  that  assumption,  it  follows  that  some 
mental  acts  or  different  numbers  of  them  will  approach  closer  to  or 
further  from  the  finite  capability  than  will  others.  The  closer  a 
"mental  act"  comes  to  the  limit,  the  greater  the  "mental  workload." 

Some  models  are  useful  as  guides  to  conceptual  thinking  about 
mental  workload;  some  are  strictly  utilitarian;  most  fall  somewhere 
in  between.  The  models  have  been  constructed  both  for  their  theoretical 
utility  and  the  applicability  to  real  world  problems.  When  one  attempts 
to  apply  a  model  to  a  real  system,  there  is  immediately  generated  a 
demand  for  data  about  the  system,  its  mission,  and  the  role  to  be 
played  by  the  human  operator (s)  in  it.  Unfortunately  it  is  difficult 
to  obtain  the  data  in  a  form  which  will  permit  direct  calculation  of 
workload  as  a  function  of  time  throughout  the  mission.  There  arc  a 


number  of  reasons  for  this  difficulty  and  each  difficulty  can  be  dealt 
with  only  at  some  cost,  since  they  stem  from  fundamental  characteristics 
of  systems,  missions  and  human  operators. 

Obviously  a  system  with  no  mission  has  no  definable  workload 
since  there  is  no  demand  on  the  human  operator  that  any  particular 
act  be  performed  at  any  particular  time  with  any  particular  precision 
or  accuracy.  The  human  operator  can  do  things  when  he  wishes,  so 
that  he  can  adjust  his  load  at  will.  It  is  clear,  also,  that  if  the 
system  itself  is  not  suitably  characterized,  there  can  be  no  definable 
workload  since  there  can  be  no  definition  of  what  is  to  be  done. 

Lastly,  the  capacities  of  the  human  operators  must  be  specified  since 
it  is  clear  that  the  load  imposed  on  two  human  operators  of  different 
capacities  or  training  will  be  different  even  though  the  demands  of 
the  system  and  the  mission  are  the  same  for  the  two. 

Thus  for  good  calculation  we  must  specify  what  is  to  be  done 
at  each  moment  of  time,  with  what  accuracy  and  precision  (the  demands) , 
and  by  whom  (the  capacities  to  perform) . 

Models 

Although  there  are  many  models  of  workload,  they  can  be  broadly 
classified  as  being  either  statistical  or  causal  in  nature.  A 
statistical  model  is  usually  concerned  with  ratios  of  system  demand 
to  operator  capacity  aggregated  ovc  some  significant  period  of  time'. 

A  causal  model  is  concerned  with  demands  at  ovry  instant  of  time. 
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An  example  of  a  statistical  model  (and  calculation  stemming  from 
it)  is  the  scanning  model  of  Senders  (2)  .  In  this  model  the  signal 
characteristics  of  the  various  stimulus  displays  are  used  directly  in 
the  calculation  of  the  frequency  and  duration  of  eye  fixations  which 
must  be  made  in  order  to  perform  the  monitoring  task  of  the  defined 
mission.  The  output  of  such  a  calculation  is  an  estimate  of  the  load 
imposed  (the  percentage  of  available  time  which  must  be  spent  looking) 
and  of  those  mission  phases  wherein  transient  overload  may  occur  either 
as  a  result  of  calculated  loads  greater  than  unity,  per  so,  or  as  a 
consequence  of  the  queueing  nature  of  the  process  modelled.  However, 
the  model  does  not  say  when  an  event  will  occur;  it  deals  only  with 
probabilities  in  time. 

A  causal  model  would  examine  every  transaction  between  operator 
and  machine  and,  for  example,  compare  the  time  required  for  its 
accomplishment  with  the  time  available  before  the  next  transaction 
occurred.  Any  of  the  single  channel  models  (Broadbent  (3)  for  example) 
is  of  this  sort. 

The  demands  made  upon  the  systems  analyst  are  quite  different 
from  the  two  kinds  of  model.  The  statistical  model  is  based  on  the 
"transfer"  of  input  statistics  to  output  statistics.  The  causal  model 
is  based  on  the  "transfer"  of  input  stimulus  to  output  action.  Since 
in  any  mission  there  are  apparent  great  differences  in  load  at  various 
times  within  the  duration  of  the  mission,  the  calculations  must  be 
separately  made  for  these  times.  The  principal  difference  is  that  for 
the  statistical  models,  the  data  cover  an  entire  period  of  (approximate) 


uniformity  of  demand  whereas  for  the  causal  model  the  data  must  be 
acquired  for  intervals  of  time  short  compared  to  the  duration  of 
mission  phases. 

The  Fine  Structure  of  Work 

How  finely  must  time  be  divided  for  the  calculation  of  workload? 
There  are  two  controlling  factors:  the  time  used  by  a  human  operator 
to  perceive,  interpret,  and  respond  to  stimuli,  and  the  intervals 
at  which  signals  are  presented  and  responses  required.  We  will  call 
the  former  "human  time"  and  the  latter  "systemic  time."  Human  time 
for  some  elementary  actions  can  be  determined  from  the  scientific 
literature  and  lead  one  to  the  notion  that  a  mission  description  at 
intervals  Of  0.5  second  would  be  sufficiently  fine  to  permit  the  use 
of  a  causal  or  moment-to-moment  model.  However,  the  descriptive  task 
of  the  analyst  is  enormous  if  the  mission  is  long — 7200  elements  per 
hour.  Further  it  may,  for  reasons  to  be  presented,  be  impossible  to 
define  the  mission  and  the  demands  on  the  operator  to  this  degree  of 
temporal  precision. 

Let  us  consider  element  i  of  a  sequence  of  actions  required  of 

the  operator  during  a  mission.  Input  i_  appears  as  a  signal  at  some 

time  T.  =  t.  +  dt  ,  where  t.  is  the  nominal  instant  when  input  i 
1  r  —  s  i  — 

should  occur,  and  dt  the  standard  deviation  of  its  actual  time  of 

s 

occurrence,  since  there  is  always  some  uncertainty  as  to  when  an  input 
will  occur.  This  uncertainty  arises  both  from  the  fact  that  some 
inputs  are  externally  developed  and  from  the  fact  that  variation  in 
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machines  and  operators  will  induce  cumulating  uncertainty  in  the 
timing  of  events.  Output  _i  is  the  action  of  the  ox^erator  in  response 
to  input  .L.  There  will,  for  any  time  of  input  be  uncertainty  of 
time  of  output  i^  and,  in  general,  output  i  will  occur  at 
2  2  1/2 

T.  _+  (dt  +  dt  )  ,  where  dt  is  the  standard  error  of  output 

1  S  d  9 

timing. 

It  will  further  be  the  case  that  the  output  i^  does  not  have  a 
required  precise  time  of  performance.  That  is  to  say,  there  is  some 
flexibility  allowed  the  operator  to  perform  when  it  is  convenient. 

Of  course  this  flexibility  is  limited  else  there  would  be  no  demand 
at  all  from  input  _i  for  any  action  at  all.  There  will  be  either  a 
range  of  acceptable  delays  between  input  i^  and  output  i_,  or  a  range 
of  acceptable  times  when  output  must  be  performed.  These  are  quite 
different  in  effect.  The  former  allows  the  timetable  of  the  mission 
to  slip,  the  latter  imposes  an  overall  timetable  but  with  less  than 
complete  internal  constraint.  Thus,  if  a  transmitter  frequency  is 
to  be  changed,  there  is  usually  a  broad  interval  within  which  mission 
accomplishment  is  indifferent  as  to  the  time  of  change,  with  respect 
to  probability  of  success.  Thus  systemic  time  may  be  very  coarse 
indeed.  In  general  we  would  expect  it  to  be  coarser  than  human  time. 
Also,  in  general,  the  coarser  the  systemic  time  the  less  short-term 
workload  is  generated  since  the  operator  has  freedom  to  move  actions 
around  to  fit  into  slots  of  available  time  subject  to  the  memory 
load  which  may  be  imposed  by  the  transient  nature  of  the  input  events. 

As  a  consequence  of  the  logic  of  the  foregoing  it  is  necessary' 
for  us  to  modify  our  earlier  simplistic  notions  about  the  workload 
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imposed  on  a  human  operator  by  the  demands  of  the  system  and  its 
mission.  It  remains  true,  of  course,  that  the  load  will  increase 
with  the  number  of  demands,  and  with  an  increase  in  the  intensity  of 
demand,  cet.  par.,  but  it  is  also  true  that  number  and  intensity 
(or  complexity)  are  conditional  upon  systemic  time  constraints. 

The  Pacing  of  Tasks 

The  notion  of  time  constraint  forces  us  to  examine  the  concepts 
of  self-paced  and  machine-paced  tasks. 

Traditionally  there  have  been  two  dichotomous  categories  of 
task:  the  self-paced  and  the  machine-paced.  The  se] f-paced  task  is 

one  in  which  the  next  action  to  be  performed  cannot  be  demanded  until 
the  previous  action  is  complete.  It  does  not  preclude  the  appearance 
of  the  next  or  even  of  a  large  number  of  stimuli  in  succession  or 
simultaneously;  it  is  only  the  output  which  is  at  the  discretion  of 
the  human  operator  (IIO)  .  The  HO  therefore  may  choose  how  fast  he  is 
to  work,  how  much  time  he  may  take  on  each  action  and  so  on. 

The  machine-paced  task  is  one  in  which  actions  are  demanded  by 
the  task  irrespective  of  whether  the  human  operator  has  performed  an 
antecedent  task  component.  Thus  it  is  possible  for  the  operator  to 
fall  behind  if  the  demands  are  beyond  his  capacity,  or  that  portion 
of  capacity  which  he  has  allocated  to  the  task. 

If  one  examines  these  two  classes  carefully,  it  becomes  clear 
that  real  tasks  have  components  of  both  in  almost  every  case.  Self- 


paced  tasks  in  a  real  situation  may  in  fact  have  ultimate  bounds  of 
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freedom  as  to  time  of  performance  even  though  there  is  no  short  term 
boundary.  An  act  may  not  need  to  be  done  "right  now"  or  at  any 
particular  time,  but  it  may  have  to  be  done  some  time.  Such  a  task 
is  by  no  means  a  pure  self-paced  form.  Machine-paced  tasks  in  real 
life  may  permit  neglect  if  circumstances  demand  it  so  that  the 
operator  may  decide  not  to  perform  or  may  miss  the  opportunity  to 
perform  without  complete  failure  of  the  mission.  Such  a  task  is  by 
no  means  machine-paced,  and  its  position  on  the  dimension  from  self- 
to  machine-paced  depends  on  the  constraint  on  time  of  performance 
specified  by  the  mission  and  the  system  involved.  Acts  (or  task 
components)  nearer  the  zero  constraint  end  may  be  done  ad  libitum  or 
neglected  altogether.  Tasks  nearer  the  other  end  must  be  done  when 
the  system  demands  them  or  neglected  only  at  great  cost  to  the  success 
of  the  mission. 

The  load  placed  on  an  operator  by  a  task  is  clearly  a  function 
not  only  of  what  is  to  be  done  in  response  to  what  stimulus  but 
also  of  when  and  with  what  time-constraint  it  must  be  done.  Furthermore, 
the  calculation  of  the  workload  of  a  task  with  a  low  degree  of  time 
constraint  may  be  a  prohibitively  expensive  chore  due  to  the  basic 
uncertainty  as  to  what  is  to  bo  done  when. 

Another  dimension  of  description  of  task  components  is  that  of 
serial  ordering.  Some  task  components  must  be  performed  in  strict 
order;  others  may  bo  performed  in  any  order.  Here  again,  if  the  degree 
of  "order  freedom"  of  a  set  of  task  components  is  high,  then  the 
calculation  of  the  workload  associated  with  them  may  be  difficult  due 
again  to  uncertainty  as  to  what  is  next  done  and  when. 
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Thus  self-paced  and  machine-paced  as  categories  are  inadequate 
and  the  degrees  of  time-constraint  and  of  order-constraint  must  be 
substituted.  A  task  component  may  be  represented  as  a  point  in  two 
dimensional  space  as  suggested  in  Figure  1 . 

The  Analysis  Problems 

The  problem  exposed  by  these  theoretical  notions  is  as  follows. 
Since  time  pressure  is  one  of  the  most  common  forms  of  load  on  a 
human  operator,  the  degree  of  time  and  order  constraint  markedly 
influences  the  load  imposed  by  a  task  component  or  a  group  of  such 
components.  The  operator  is  free,  to  varying  degrees,  to  move  acts 
from  moments  of  high  demand  to  moments  of  low  demand.  To  the  extent 
he  is  able  to  do  this  efficiently,  he  can  avoid  overload  subject  to 
the  limitations  of  his  memory  for  those  acts  which  must  be  serially 
ordered  either  in  whole  or  in  part. 

Thus  it  can  be  seen  (as  we  see  it)  that  there  can  be  no  solution 
to  the  workload  analysis  based  on  molecular  examination  of  human 
performance.  The  difficulty  stems  directly  from  the  (literal) 
immensity  of  the  calculation  required  to  arrive  at  the  actual  sequence 
of  acts  which  minimizes  the  load  placed  on  the  human  operator  and 
which  a  skilled  HO  will  tend  to  asymptotically. 

A  way  out  can  be  found  by  expanding  the  time  unit  of  analysis 
and  the  problem  is  that  of  determining  how  large  that  unit  is  to  be. 
There  are  many  alternative  approaches. 
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One  could  use  an  arbitrary  size  of  time  slot  and  require  that 
all  acts  will  be  performed  within  that  time.  Then  within  the  time 
slot  all  acts  would  be  aggregated  and  the  load  computed  as  an  average 
over  the  duration  of  the  time  slot.  This  has  the  advantage  of 
greatly  simplifying  the  calculation.  However,  it  may  inflate  the 
estimate  of  load  and  be  unduly  conservative  since  it  restricts  the 
freedom  of  time  of  performance. 

Another  alternative  would  use  a  time-varying  time  slot  which 
is  large  in  periods  of  relatively  low  demand  and  small  in  periods  of 
relatively  high  demand.  A  useful  method  might  be  one  which  had  time 
slots  in  which  equal  numbers  of  acts  were  demanded.  The  advantage 
would  be  a  relatively  uniform  calculational  effort  per  time  slot. 

In  either  case,  there  might  be  acts  which  had  such  lack  of 
time  constraint  that  they  could  be  performed  at  any  moment  over  a 
relatively  long  span.  These  acts  would  have  to  be  treated  in  a 
separate  way  and  used  as  fillers  for  those  time  slots  which  were 
singularly  low  in  load  and  fell  within  the  larger  time  span  of  these 
unconstrained  acts.  Arbitrarily,  any  act  with  a  constraint  so  low 
that  it  could  be  performed  during  a  span  of  time  greater  than  two 
time  slots,  would  be  designated  a  "time-free  act"  and  assigned  to 
the  slot  within  its  reach  with  the  lowest  load. 

Since  for  any  of  these  schemes  there  would  be  variation  of 
calculated  load  with  variation  of  length  of  time  slot  it  would  be 
desirable  to  have  a  program  which  would  calculate  load  as  a  function 
of  slot  length  (up  to  the  limits  determined  by  calculational  demands) . 
In  all  cases,  the  determination  of  the  length  of  the  time  slots,  and 
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the  character  of  the  time-free  acts  would  depend  on  the  detailing 
of  the  mission  on  a  fine  time  basis  and  the  specifying  for  each  act 
of  its  serial  time  dependency  and  time  constraints. 

The  Task  De s cription  Problem 

Workload  cannot  be  calculated  without  a  detailed  description 
of  events  in  the  mission  as  a  function  of  time.  This  has  been 
evident  from  the  very  first  attempts.  Lindquist  and  Gross  (4)  used 
a  "Second-by-Second  Operational  Analysis"  to  estimate  the  workload 
placed  on  the  astronaut  by  the  first  orbital  mission  in  the  Mercury 
spacecraft.  Most  of  the  workload  calculations  were  made  on  the  basis 
of  Information  Theory  (5)  or  on  the  basis  of  Sampling  Theory  of  Visual 
Attention  (6) .  There  could  not  in  the  ordinary  course  of  events  be 
any  validation  of  the  numbers  generated  other  than  the  success  or 
failure  of  the  mission  and  this  was  a  rather  unsatisfactory  validation 
at  best. 

Siegel  et  al  used  a  simulation  technique  for  the  human 
operator  (HO)  performing  mission  elements  and  introduced  stochastic 
variation  into  the  performance  of  the  simulated  HO.  Then,  by  the  use 
of  Monte  Carlo  methods,  they  were  able  to  generate  distributions  of 
probability  of  success  of  the  mission  as  a  function  of  variation  in 
the  task.  This  method  had  the  distinct  advantage  over  the  Lindquist 
and  Gross  approach  of  using  the  full  range  of  human  performance  instead 
of  mean  data  taken  from  the  literature.  Again,  however,  validation 


was  difficult. 
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Both  of  the  foregoing  attempts  did  not  consider  the  time 
lability  of  events — signals  and  required  actions  on  the  part  of  HO. 

As  a  result  both  probably  overestimated  the  load  to  be  imposed. 

The  queueing  models  of  Senders  and  Posner  (6),  Carbonelle  (7), 
Rouse  (8) ,  and  others,  generally  are  based  on  the  assumption  of 
machine-paced  task  structure.  The  demands  come  along  and,  if  two  or 
more  occur  at  once,  there  is  a  transient  overload,  or  the  sytem  has 
to  wait  for  the  action  demanded  while  the  one  chosen  is  dealt  with. 

If  the  system  is  tolerant  or  absorbent  of  delay,  then  there  is  no 
inherent  problem  with  that  state  of  affairs.  It  is  only  when  the 
action  may  not  be  delayed,  or  has  a  definite  period  of  acceptance 
with  non-absorption  after  that,  that  true  overload  can  occur  from  the 
simultaneous  demand.  The  systems/misfii on  analyst  has  responsibility, 
therefore,  for  specifying  which  demanded  actions  have  time  lability 
and  to  what  extent,  if  the  results  are  to  be  reasonable. 

There  are  other  problems  in  description.  These  stem  from  the 
fact  that  much  of  what  HO  does  is  not  in  response  to  a  definabl e 
stimulus  (if  indeed  to  any  stimulus  at  all).  Thus  HO  may  think, 
daydream,  plan  ahead,  consider  alternatives  and  so  on.  The  Table  1 
taken  from  Johannsen  and  Rouse  (9)  is  an  example  of  a  description  at 
a  level  which  prohibits  the  assignment  of  load  based  on  performance 
data.  Here  the  elements  of  behaviour  are  not  those  which  psychologists, 
for  obvious  reasons,  have  spent  their  time  in  the  laboratory  studying. 
They  are  intrinsically  difficult  to  deal  with,  to  define,  to  measure. 
Yet  at  one  level  of  discouse  they  most  precisely  describe  what  is 
being  done  however  inconvenient  for  the  missions  analyst  it  may  be. 
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Table  1.  Protocol  for  Typical  City  Trip 


INSERT  KEY  IN  IGNITION 
PUT  ON  SEAT  BELT 

PRESS  GAS  PEDAL  TO  FLOOR  AND  ALMOST  TOTALLY  RELEASE 
TURN  KEY 

LISTEN  FOR  ENGINE  SOUND 
IF  SO,  THEN  GIVE  CAS 
ELSE,  STOP  AND  GO  BACK  TO  TURN  KEY 
WAIT  FOR  CAR  TO  WARM  UP — DAYDREAM 

LOOK  AROUND— SEE  IF  I  CAN  BACK  Ur  OKAY— INCLUDES  USING  MIRRORS 
IF  SO,  THEN  PUT  CAR  IN  REVERSE 
ELSE,  WAIT  FOR  ALL  CLEAR 
PUT  RIGHT  ARM  ON  REAR  BACK  SO  AS  TO  SEE  BETTER 
STEER  WITH  LEFT  ARM,  ACCELERATE  AND  BACK  ONTO  STREET 
DETERMINE  WHEN  CLFAR  TO  GO  FORWARD — STOP  BACKING  UP — PRESS  BRAKE 
PUT  CAR  IN  DRIVE 

LOOK  AROUND — SEE  IF  I  CAN  PROCEED 
IF  SO,  ACCELERATE 
ELSE,  WAIT  FOR  ALL  CLEAR 

LIMIT  SPEED  SINCE  STOP  SIGN  COMING  UP- -CONTINUE  IOOKING  AROUND 
STEER  SO  AS  TO  STAY  "SORT  OF"  IN  LANK 

ESTIMATE  DISTANCE  TO  STOP  SIGH- -CHECK  FOR  TIME  TO  DECELERATE 


IF  SO,  REMOVE  FOOT  FROM  GAS  AND  OVER  TO  BRAKE 

EliSE,  UPDATE  ESTIMATE  OF  DISTANCE- -CONTINUE  LOOKING  AROUND/STEERING 
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Table  1  (continued) 

TURN  ON  LEFT  DIRECTIONAL 

WHEN  FAIRLY  CLOSE  TO  STOP  SIGN,  PUSH  BRAKE  HARDER  AND  STOP 
LOOK  LEFT  AND  RIGHT  FOR  TRAFFIC 

IF  ALL  CLEAR,  TURN  LEFT  AND  ACCELERATE 

ELSE,  WAIT  FOR  ALL  CLEAR  AND  CONTINUE  UPDATING  ESTIMATES 
STRAIGHTEN  OUT  SO  AS  TO  KEEP  "SORT  OF"  IN  LANE 
ACCELERATE,  BUT  NOT  TOO  MUCH  BECAUSE  STOP  SIGN  COMING  UP 
LOOK  AROUND  AT  TRAFFIC — ALSO  AT  HOUSES  AND  YARDS — DAYDREAM 
EXECUTE  STOP  SIGN  ROUTINE— ONE  FOR  STOPPING— ONE  FOR  STARTING 

—USE  FOUR-WAY  STOP  SIGN  ROUTINE 
EXECUTE  ENROUTK  ROUTINE- -INCLUDING  TALKING,  SIGHTSEEING,  ETC. 


PLAN  ROUTE — WHAT  STREETS  TO  TAKE 


EXECUTE  STOP  SIGN/STOP  LIGHT/TURNING/PASSING/LANE  CHANGING  ROUTINES 


LOOK  AROUND  FOR  APPROPRIATE  PARKING  SPACE 

IF  ONE  FOUND,  DETERMINE  PRAM  FOR  GETTING  INTO  IT 
ELSE,  CONTINUE  IOOKING  AROUND  AND  STEERING 


Table  1  (continued) 

EXECUTE  PLAN  OPEN-LOOP,  WITH  FINAL  UPDATES  AS  ERRORS  CAN  BE  ESTIMATED 
PUT  CAR  IN  PARK 

TURN  OFF  RADIO,  HEATER,  ETC.  IF  APPROPRIATE 
TURN  OFF  KEY 


REMOVE  KEY 


Can  those  high  level  global  acts  be  broken  down  in  a  reasonable- 
way  into  more  elemental  acts?  We  might  do  the  following: 

Insert  Key  in  Ignition 

Identify  from  memory  the  location  of  key;  control 
hand  (of  choice)  toward  location  of  key,  (using  closed 
loop  as  well  as  open  loop  control) ,  grasp  key,  tactually 
examine  orientation  of  key,  correct  oricnLation  (closed 
loop  either  tactual  or  visual  or  both) ,  direct  key  toward 
lock  (using  closed  loop  control) ,  insert  key  into  lock 
(using  closed  loop  control  tactual  this  time) ,  test 
whether  key  is  "home"  (tactual  feedback) . 

We  have  now  reduced  the  act  of  inserting  key  to  a  series  of 
more  elemental  acts  some  of  which,  at  least,  have  been  studied  in  the 
laboratory  and  for  which  data  exist  or  can  be  inferred.  One  can  go 
further.  Consider  the  elemental  act:  direct  key  toward  lock. 

This  can  be  further  broken  down  into  a  time  sequence  of  error 
detections  of  disparity  between  plan  and  actuality,  and  the  emission 
of  correcting  impulses  to  the  appropriate  muscles  to  reduce  the 
error.  The  mathematics  of  transfer  function  analysis  can  be  applied 
to  predict  time  and  error. 

The  demand  that  "key  be  inserted  in  ignition"  is  not  quantifiabl 
without  a  part  task  simulator  involving  keys  and  locks  and  subjects. 

But  it  is  precisely  these  that  we  wish  to  eliminate  from  the  workload 
analysis  process.  To  overcome  this  we  have  broken  down  the  insertion 
of  the  key  into  smaller  parts.  Now,  i f  we  hud  a  sufficiently 


detailed  description  of  the  actual  location  of  the  key,  its  shape, 
the  size  of  the  keyhole,  the  presence  or  absence  of  guiding  marks 
around  the  keyhole,  the  illumination  atound  the  keyhole,  the  force 
involved  in  inserting  the  key,  and  so  on,  we  could  begin  to  derive 
man-machine  data  from  our  knowledge  of  man  and  our  description  of  the 
machine. 

The  following  steps  are  required  for  application  of  the  model: 

1)  A  description  of  the  task  as  a  series  of  elemental 
acts  in  units  of  time  smaller  than  the  least 
significant  unit  of  time  of  human  performance. 

2)  A  statement  of  the  time-lability  and  order-constraint 
of  each  of  the  ordered  task  elements. 

3)  A  classification  of  each  task  element  in  terms 
of  an  acceptable  taxonomy  of  human  performance. 

4)  A  statement  of  the  expected  capacity  of  the  human 
operator  to  perform  that  taxonomic  element. 

5)  A  statement  of  the  expected  level  of  demand  for 
that  element. 

6)  The  calculation,  for  each  unit  of  time,  of  the 
load  imposed  with  that  time  unit. 

7)  A  calculation  of  the  probability  of  interference 
between  task  elements,  derived  from  the  queueing 
model  previously  described. 
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8)  A  calculation  of  the  resultant  human  failure 

and  error  rates,  derived  from  the  queueing  model. 

In  order  to  obtain  the  information  needed  for  step  1,  it  would 
be  necessary  to  construct  a  mission  description  which  stated,  for 
each  unit  of  time,  what  had  to  be  done,  with  what  precision,  and  with 
what  time  constraints.  Obviously,  if  the  human  operator  may  perform 
a  task  element  any  time  in  a  period  of  ten  minutes,  that  element  is 
less  contributing  to  load  and  intei-ference  than  if  the  same  element 
had  to  be  performed  within  a  period  only  ten  seconds  in  duration . 

When  time  constraints  are  great,  the  probability  of  queueing  problems 
and  consequent  interference  and  error  is  increased.  It  is  clear 
that  load  and  performance  are  strongly  specified  by  the  specification 
of  the  mission;  if  the  latter  is  not  properly  done,  then  the  former 
cannot  be  properly  done. 

In  general,  it  is  this  kind  of  process  that  is  undertaken  for 
a  reductionist  analysis.  There  must  be  a  MISSION  DESCRIPTION  which 
can  be  mapped  into  a  MAN-MACHINE  PERFORMANCE  encyclopedia  which  derives 
from  a  SYSTEM  DESCRIPTION  which  interacts  with  a  HUMAN  OPERATOR 
PERFORMANCE  encyclopedia.  HO  performance  data  combined  with  the 
physical  description  of  the  system  (the  work  environment)  yields 
man-machine  performance  data.  The  mission  description,  when  "forced" 
through  the  man-machine  system,  yields  a  sequence  of  transactions 
and  processes  required  throughout  the  course  of  the  mission. 
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All  of  these  elements  are  necessary  for  analytical  solution 
of  workload  by  any  model  for  any  real  system  doing  a  mission.  The 
critical  question  is  whether  such  a  solution  is  achievable  and 
meaningful . 


SOME  EARLY  EXAMPLES  OF  ANALYSIS 

Lindquist  and  Gross  (4) 

Some  missions,  with  some  systems,  are  by  their  nature  easy  to 
analyze.  Where  the  mission  designer  has  precisely  stipulated  the 
sequence  of  events;  where  there  are  no  free  periods  of  time  left 
unaccounted  for;  where  there  are  no  "unexpected"  events;  a  precise 
sequence  of  elemental  acts  and  the  loads  associated  with  them  can  be 
constructed.  The  first  Mercury  capsule  flights  were  of  this  nature. 

An  analysis  was  made  of  this  mission  by  Lindquist  and  Gross  (4)  using 
a  technique  newly  developed  for  the  purpose.  The  underlying  assumptions 
were;  every  demand  made  by  the  system  had  to  be  satisfied  as  rapidly 
as  possible  by  the  HO  (astronaut  in  this  case) ;  loads  imposed  by 
overlapping  elements  of  demand  were  strictly  additive;  no  loads 
were  generated  by  sequences  of  activities  per  se  other  than  the  loads 
associated  with  those  activities.  The  entire  mission  was  described 
by  a  simple  tabulation  of  every  event  that  occurred  (or  was  supposed 
to  occur)  during  each  second  from  prelaunch  to  landing.  Then  for 
each  of  these  events  a  load  figure  was  assigned  based  on  whatever 


laboratory  data  were  available  from  the  literature.  In  some  cases 
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ad  hoc  experiments  were  run  to  determine  loads.  In  some  cases  the 
measure  of  load  would  simply  be  the  proportion  of  the  time  available 
for  the  activity  actually  required  for  performance. 

The  sum  of  the  loads  in  each  second  was  found  and  the  result 
plotted  as  a  function  of  mission  time.  Where  (if  at  all)  the  load 
exceeded  1.0,  adjustments  of  events  would  have  to  be  made.  Where  the 
load  was  close  to  the  1.0  level,  adjustment  would  also  be  made  but 
the  urgency  of  the  adjustments  was  a  function  of  the  level,  of  course. 
Certain  high  load  intervals  wore  found  and,  where  possible,  relieved 
by  judicious  moving  of  events  to  other  time.  No  provision  was  made 
in  the  analysis  for  flexibility  of  the  mission  and  the  system  in 
allowing  events  to  be  done  on  an  ad  lib  basis  within  certain  temporal 
boundaries.  No  variability  data  were  entered  into  the  solution  and 
none  came  out.  There  was  no  way  to  estimate  the  probability  that  an 
overload  would  occur,  nor  whether  a  load  of  0.9  was  acceptable  (other 
than  thiit  it  was  less  than  1.0)  . 

Siegel  and  Wolf  (5) 

The  method  of  Siegel  and  Wolf  involves  a  simulation  model 
realized  on  a  digital  computer.  The  model  works  its  way  through  a 
mission  in  a  succession  of  discrete  time  intervals  and  calculates, 
using  Monte  Carlo  techniques,  the  time  taken  for  the  performance  of 
a  microelement  of  the  mission.  The  time  remaining  for  the  accomplishment 
of  all  waiting  events  is  recalculated,  "stress”  levels  calculated, 
distributions  altcre.d  on  the  basis  of  the  "stress"  and  the  next  time 
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interval  entered.  Then  the  process  is  repeated  for  the  next  event 
needing  to  be  done.  The  program  is  capable  of  generating  distributions 
of  the  outcomes  of  the  successive  runs  and  assigning  a  success  or 
failure  to  each.  For  some  runs,  of  course,  the  time  will  have  run 
out,  and  the  run  termed  a  failure,  because  of  the  random  nature  of 
the  selection  of  operating  times  from  specified  distributions.  The 
models  have  been  extended  to  include  more  than  one  operator  and 
machine  systems  of  varying  complexity. 

Wherry  (10) 

Even  more  sophisticated,  more  complex,  and  more  detailed  is 
the  Human  Operator  Simulator  (HOS)  described  by  Wherry  (10)  .  HOS  is 
an  active  encyclopedia  of  human  performance  data  against  which  the 
requirements  of  a  task  can  be  played.  What  come  out  are  "time  to 
perform"  and  "error  probabilities."  Here  such  simple  acts  as 
reaching  for  a  knob,  grasping  it,  and  then  turning  it  are  separately 
analyzed  and  quantified.  Naturally  the  system  demands  a  very  low 
level  of  description,  highly  detailed  and  matched  to  the  data  base. 

This  system  is  a  first  approximation  to  the  generalization  made  on 
page  19. 

A  practical  analysis  technique  must  have  aspects  of  all  three 
predecessors:  the  fine  time  structure  of  Lindquist  and  Gross,  the 

Monte  Carlo  method  of  Siegel,  and  the  store  of  human  performance  data 
of  HOS  along  with  the  means  to  attach  the  data  to  the  task.  This 
last  is  of  the  greatest  significance. 
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Comment 


Because  of  the  non-analytic  nature  of  the  description  which 
is  required,  it  is  almost  surely  the  case  that  the  solution  to 
practical  problems  will  come  from  algorithmic  rather  than  from  foimal 
methods.  In  general,  models  must  be  built  of  the  machine  and  t.he  HO. 
These  two  must  have  a  set  of  rules  of  interaction,  the  man-machine 
interaction.  Finally,  there  must  be  a  way  of  introducing  information 
about  the  mission  into  the  model  in  order  to  derive  estimates 
relevant  to  particular  practical  problems. 

Earlier  we  indicated  that  one  of  the  chief  features  of  the 
time-based  description  of  a  mission  is  that  events  are  not  serially 
ordered  in  time.  For  each  class  of  events,  and  for  each  event  within 
a  class,  there  will  be  some  degree  of  flexibility  of  timing  which 
will  vary  as  well  with  the  point  in  a  mission  that  the  event  occurs, 
the  skill  of  the  operator,  the  presence  or  absence  of  other  events 
and,  in  all  probability,  many  other  factors  as  well.  Therefore  for 
any  event  a  large  number  of  descriptors  is  required.  Among  these 
are:  the  degree  of  serial  constraint,  the  degree  of  time  constraint, 

its  classification  as  transaction  or  process,  its  classification  as 
being  a  response-dependent.,  a  stimulus-dependent,  or  an  independent 
stimulus;  its  being  a  response-dependent,  a  stimulus-dependent,  or 
an  independent  response. 

The  following  analysis  is  a  restructuring  of  the  analysis 


task  which  includes  all  of  these  factors. 
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The  Analysis  of  Speeded  Operation 

Workload  is  of  consequence  when  the  110  is  involved  in  a  more 
or  less  prolonged  mission  of  which  some  aspects  must  be  performed 
within  a  limited  period  of  time.  This  circumstance  may  be  called 
speeded  operation.  The  units — some  of  which  may  overlap — into  which 
speeded  operation  may  be  analyzed  for  the  purpose  of  assigning  demands 
or  deadlines,  on  the  one  hand,  and  capacities  or  performance  times, 
on  the  other,  may  be  called  events .  An  event  is  the  coupling  of  a 
response  by  HO  with  its  eliciting  circumstance.  Some  functional 
events  may  easily  be  described  as  transactions  between  a  specified 
environmental  stimulus  and  a  response  that  is  completely  determined 
by  that  stimulus.  For  other  events  there  is  either  no  determinate 
stimulus  or  a  response  that  will  vary  for  the  same  stimulus  as  a 
consequence  of  states  of  other  stimuli  or  the  effects  of  recent  or 
remote  experiences  of  the  HO.  Such  an  event,  then,  must  be  regarded 
as  an  internal  process  of  the  HO,  including  the  culminating  response. 

Definitions,  Examples,  a nd  Characteristics  of  the  Classes 
of  Events 

1.  A  Stimulus-Response  (SR)  Transaction  is  defined  as 
one  in  which  the  stimulus  occurs  independently  in  time  of 
preceding  responses.  Some  familiar  tasks  made  up  of  such 
transactions  are  tracking  and  typing  from  dictation.  Tasks 
of  this  kind  are  often  called  forced  pace.  There  may  be  more 
or  less  sequential  dependency  in  the  sequence  of  stimuli. 
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with  successive  letters  in  words  to  be  iyped  illustrative 
of  high  sequential  dependency  and  a  sequence  of  stimuli  for 
various  emergency  responses  illustrative  of  low  sequential 
dependency.  Typically,  there  is  a  buit-in  time  constraint 
in  making  responses,  as  a  stimulus  may  be  forgotten  as  new 
stimuli  occur.  However,  time  constraint  may  range  from 
excessively  tight  as  in  tracking  to  very  flexible  as  in 
turning  down  the  volume  of  a  headset.  In  addition,  there  is 
more  or  less  serial  constraint,  the  degree  to  which  the 
sequence  of  responses  must  match  the  sequence  of  stimuli. 

2.  A  Response-Stimulus-Response  (HSR)  Transaction  is 
defined  as  one  for  which  the  stimulus  occurs  or  becomes 
appropriate  only  after  a  preceding  response.  One  example  is 
typing  from  text.  Another  is  performing  on  the  Senders 
bit-box  where  a  new  signal  for  response  occurs  on  the  making 
of  the  previous  response.  Tasks  made  up  of  such  transactions; 
are  often  called  self  paced  as — in  contrast  with  forced-pace 
behaviour — the  time  pressure  is  indirect,  depending  on 
instructions,  over-all  deadlines,  etc.  Responses  are  serially 
constrained  as  there  is  no  stimulus  for  response  until  the 
preceding  response  has  been  made.  Although  it  is  typical 
for  signals  to  be  permanent  (as  on  a  printed  page)  or  to 
persist  until  a  response  is  made  (as  in  the  bit-box)  this 
is  not  necessarily  the  case.  For  example,  the  bit -box  could 
be  modified  to  produce  a  transient  signal. 
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3.  A  Process  is  defined  as  an  event  for  which  there  is 
no  single  identif iable  signal  that  elicits  the  response  but 
rather  some  internal  activity  of  the  HO.  Examples  include 
all  cases  where  the  response  is  made  on  the  basis  of  a  plan, 
a  remembered  instruction,  or  a  combination  of  preceding 
stimuli  or  responses.  Such  responses  often  occur  at  an  upper 
level  of  the  response  hierarchy,  where  they  take  the  form  of 
decisions.  It  is  typical  for  there  to  be  flexible  time  and 
serial  constraint  for  these  responses. 

Properties 

For  the  modeling  of  workload,  transactions  and  processes 
need  to  be  more  fully  described  as  they  are  profoundly  affected 
by  the  functional  properties  of  the  stimuli,  of  the  responses 
themselves,  and  of  the  relation  between  stimuli  and  responses. 
Some  of  these  properties  were  alluded  to  in  the  preceding 
treatment  of  the  elicitation  classes:  c.g. ,  time  constraint, 
serial  constraint,  and  sequential  dependency.  Following  are 
definitions  and  examples  of  properties: 

1.  Stimulus  Properties 

a.  Sequential  dependency  is  defined  as  the  degree 
of  necessity  for  a  particular  stimulus  to  appear  at  a 
particular  time  in  the  series  of  stimuli.  At  one 
extreme  there  could  be  an  entire  mission  that  is  described 
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in  advance.  At  the  other  extreme  there  is  a  random 
series  of  stimuli.  In  the  middle  ground  is  a  series 
of  stimuli  having  a  probabilistic  relation  to  one 
another. 


b.  Group  size  is  defined  as  the  number  of 
stimuli  not  yet  responded  to  that  are  available  to  the 
operator  in  making  a  response  to  one  of  them.  For  the 
bit  box  group  size  is  1  as  the  only  stimulus  present 
is  that  for  the  imminent  response.  In  typing  from 
text  there  is  a  large  group  of  stimuli  present  when 
making  a  response,  with  perhaps  eight,  or  ten  useful  for 
the  HO's  performance. 

c.  Duration  is  defined  as  the  length  of  time  the 
stimulus  is  presented.  In  typing  from  dictation  each 
stimulus  is  presented  momentarily.  In  the  bit  box 

each  stimulus  persists  until  a  response  is  made.  In 
typing  from  text  stimuli  usually  remain  present  even 
after  their  responses  have  been  made. 

2.  Response  Properties 

a.  Serial  constraint.  A  serially  constrained 
response  is  defined  as  one  that  will  not  be  accepted 
by  the  system  or  is  inappropriate  unless  it  is  made 
after  certain  responses  and  before  others.  In  typing 
from  text  each  response  is  serially  constrained  within 


the  series  of  responses.  In  recording  the  settings  of 


a  group  of  meters  there  is  often  no  necessary  serial 
constraint. 


b.  Time  constraint.  A  time  constrained  response 
is  defined  as  one  that  must  be  made  within  some 
specified  time  window  after  the  appearance  of  the 
signal  or  within  a  specified  phase  of  the  mission.  A 
sudden  move  of  a  target  must  be  responded  to  immediately 
in  tracking.  Answering  a  question  from  mission  control 
can  be  put  off  for  a  little  while  until  a  tracking 

error  has  been  corrected.  For  both  of  these  SR  transactions 
there  is  direct  time  constraint.  In  typing  from  text 
the  time  constant  is  indirect.  The  operator  may  be 
trying  to  type  as  quickly  as  possible,  as  quickly  as 
possible  with  no  more  than  five  percent  errors,  to 
complete  a  passage  within  six  minutes,  etc. 

c.  Hierarchical  level  is  defined  by  the  events 
subsumed  under  a  given  event.  In  the  bit  box  each 
transaction  is  at  the  lowest  hierarchical  level.  A 
decision  to  adjust  altitude  subsumes  the  transactions 
of  checking  the  altimeter  and  altitude  indicator  and 
operating  each  of  a  number  of  controls.  A  decision  to 
abort  a  mission  subsumes,  in  turn,  such  transactions 

as  the  decision  to  adjust  altitude.  It  is  characteristic 
as  hierarchical  level  increases  for  the  events  to  be 
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processes  rather  than  transactions  as  a  good  deal  of 
information  may  be  integrated  with  no  single  eliciting 
stimulus  being  identifiable. 

3 .  Stimulus-Response  Property;  Directne s s  of 
Implements  Lion 

Directness  of  implementation  is  defined  by  the 
probability  that  the  stimulus  will  instigate  a  particular 
response.  In  typing  there  is  a  defined  particular 
response  to  each  letter.  However,  a  signal  may  be 
given  to  the  operator  with  a  more  general  message: 
e.g.,  perform  a  certain  manoeuvre  if  possible.  Even 
less  directly,  the  operator  may  be  advised  of  possible 
weather  conditions  to  "keep  in  mind."  In  the  first 
case  the  stimulus  is  part  of  a  transaction.  Jn  the 
latter  cases  it  is  possible  that  it  will  form  part  of 
a  process. 

Thus  the  task  of  typing  textual  material  at  sight  is  a  serially 
constrained  (because  the  letters  must  be  typed  in  the  same  order  ns 
they  are  in  the  text) ,  time-unconstrained  (because  there  is  no 
stipulated  time  pressure) ,  sequence  of  transactions.  Typing  from 
oral  dictation  is  a  serially  constrained,  time  constrained  (since  the 
speech  goes  on  independent  of  the  typing  responses)  sequence  of 
transactions.  In  both  cases  the  events  are  response  dependent 
(since  no  response  can  bo  made  until  the  preceding  one  has  been 
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completed) .  A  common  laboratory  task  is  one  in  which  the  new  stimulus 
appears  only  after  the  preceding  response  has  been  completed,  as  in 
an  information  processing  task.  Independent  events  appear  at  random 
and  the  responses  may  be  made  in  any  order  as  convenient  to  the 
performed.  In  all  cases  these  are  transactions.  A  process  is 
exemplified  by  the  event  of  going  through  an  intersection  when  the 
light  is  yellow.  Here  the  response  is  a  function  of  a  large  number 
of  factors  including  memory  information  brought  to  the  decision  process 
by  the  performer. 

Some  of  the  possible  types  of  events  are  inherently  contradictory, 
others  are  common  in  real  world  situations.  Still  others  occur 
almost  without  exception  only  in  the  laboratory.  It  is  unfortunate 
that  many  of  the  available  data  derive  from  this  last  class  of  event. 
Both  the  classical  machine-paced  and  self-paced  tasks  are  rare  outside 
the  laboratory  situation. 


DATA  REQUIREMENT  FOR  ANALYSIS 

In  order  to  clarify  and  make  graphic  the  nature  of  the 
defining  statements  Figure  2  is  presented  to  show  a  time-line  analysis 
which  could  servo  as  the  basis  for  a  computer  program  for  the 
estimation  of  the  demand  of  a  task. 

Let  the  capacities  of  the  operator  be  C^,  C^r  and  so  on.  Let 

the  demands  of  the  system  on  these  capacities  be  D  ,  D^,  and  so  on. 


The  loads,  then,  according  to  the  Senders  model  (1)  are  defined  as 


Figure  2.  Diagram  showing  how  Time  and  Order  Constraints  arise 
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the  ratios  of  demand  to  capacity.  For  each  load  applied  to  an  elemental 
capacity  we  can  generate  a  time  line  as  shown  in  Figure  2.  When  a 
mark  is  made  on  the  line  1,  it  indicates  that  a  demand  has  arisen 
and  an  act  is  required.  The  act  may  be  as  simple  as  monitoring  an 
indicator,  or  as  complex  as  choosing  to  abort  a  mission.  The  duration 
of  each  mark  shows  the  time  constraint  associated  with  that  event. 

When  the  mark  is  long,  the  event  can  be  completed,  i.e.,  the  act 
performed,  at  any  time  over  the  length  of  the  mark.  When  the  mark  is 
short,  as  in  the  first  mark  on  line  3,  the  response  must  be  completed 
within  a  short  time.  We  can  also  specify  the  serial  constraint  of 
each  of  the  events.  For  example  the  third  event  in  line  2  has  a  3 
before  it  indicating  that  the  overlapping  event  in  line  3  must  be 
completed  before  the  act  called  for  in  line  2  can  be  completed.  In 
general,  for  any  event  which  is  serially  constrained  at  all  there  will 
be  some  linking  symbols  with  other  lines.  The  first  event  in  line  1 
has  no  symbol  indicating  that  it  may  be  performed  at  any  time  within 
its  time  band.  This  set  of  lines  and  links  is  the  first  step  toward 
the  estimation  of  the  load  imposed  by  a  mission.  The  next  step  must 
take  into  account  the  interval  over  which  the  events  are  to  be  combined 
and  treated  statistically.  In  the  figure  there  are  natural  boundaries 
which  are  apparent  to  the  eye  without  analysis.  The  arrow  along  the 
abcissa  indicates  natural  break  or  chunk  of  events.  The  sequence 
of  events  shown  by  the  figure  is  then:  2,  1,  2,  3,  2;  or  1,  2,  2,  3,  2, 
and  so  on.  With  a  large  number  of  possible  paths  through  the 
temporal  maze  the  analysis  problem  becomes  formidable . 


Still  further  information  can  be  usefully  added  to  the  figure. 


For  each  line  there  will  be  a  distribution  of  performance  times. 

The  mean  for  each  line  is  shown  as  a  length  of  bar  to  the  left  of  the 
line.  (Here  they  are  all  the  same.)  The  relationship  between  these 
times  to  perform  and  the  permissible  times  within  which  performance 
must  be  completed  provides  insight  into  the  pressure  or  time-stress 
which  is  placed  on  the  human  operator  by  each  event.  Thus  the  third 
event  in  line  2  must  be  performed  immediately  upon  its  arrival  since 
the  time  to  perform  is  virtually  as  long  as  the  acceptable  window  of 
performance.  The  same  is  true  of  the  second  and  third  events  of  line  1. 

Since  the  event  in  line  3  must  be  completed  before  the  third 
event  in  line  2,  and  since  the  event  in  line  3  takes  a  significant 
part  of  the  time  available  for  it,  there  is  a  transferred  requirement 
to  the  event  in  line  3  that  it  be  done  immediately  upon  the  stimulus 
to  the  transaction.  Otherwise  there  would  not  be  time  for  the  event 
in  line  2  to  be  completed  within  its  allotted  time. 

If  one  now  increases  this  diagram  to  include  everything  which 
might  occur  during  a  mission,  the  difficulties  both  of  description 
and  action  by  HO  can  be  appreciated.  Yet  such  a  description  is 
necessary  if  we  are  to  have  a  complete  analysis.  How  to  escape  this 
dilemma  is  a  major  problem. 

Most  models  have  been  concerned  with  transactions  even  though 
it  is  probably  the  case  that  a  highly  skilled  HO  uses  processes  far 
more  than  transactions  in  operating  complex  systems.  Process  analysis 

is  a  more  difficult  task.  A  process  may  be  determined  by  events  in 
the  future  as  well  as  those  in  the  past.  Thus  a  skilled  operator  may 
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"play  a  game"  with  the  system  and  the  mission  in  such  a  way  as  to 
minimize  the  future  load  that  will  be  imposed  on  him.  We  would  in 
fact  expect  and  hope  that  skilled  operators  would  behave  in  an  optimum 
forecasting  way  to  achieve  this  goal.  How  can  it  be  done?  The 
operator  may  use  the  techniques  of  a  chess  player  in  analyzing  the 
possibilities  of  the  system's  "next  moves"  and  choose  the  behaviour 
that  improves,  according  to  some  criterion,  the  situation  for  him 
in  the  near  or  distant  future.  The  operator,  when  confronted  with  a 
choice,  may  analyze  the  effects  of  future  demands  on  the  present 
choices  and  choose  the  act  which  leads  to  a  series  of  diminishing 
loads  or  demands.  Thus  the  first  choice  may  not  necessarily  be  the 
apparently  "best"  if  the  future  implications  are  taken  into  account. 
Then  on  each  succeeding  choice ,  one  more  step  into  the  future  is 
added  and  the  process  repeated.  Although  this  seems  like  a  complicated 
process,  it  is  no  more  than  what  a  skilled  game  player  does  in 
"rapid  transit  chess"  without  conscious  thought,  and  in  a  second  of 
decision  time.  How  can  such  a  program  be  implemented?  The  chess 
playing  programs  offer  a  solution.  Examine  all  choices,  evaluate 
each  according  to  a  set  of  criteria,  examine  the  consequences  of 
those  what  exceed  a  threshold  level,  repeat  with  all  choices  available 
at  this  level  and  so  on  until  some  horizon  is  reached.  That  sequence 
which  led  to  the  best  state  at  the  horizon  is  followed  for  one  step. 
Then  the  process  is  repeated.  This  latter  involves  only  the  addition 
of  whatever  new  choices  may  have  arisen  subsequent  to  the  first 
choice  act,  plus  those  possible  events  which  "rose  above  the  horizon" 
because  of  the  passage  of  time  in  the  mission.  Then  the  process  is 
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repeated  throughout  the  mission.  The  distance  of  the  horizon  is 
one  variable  which  corresponds  to  the  ability  of  the  operator  to 
"look  ahead";  the  criteria  by  which  various  choices  are  evaluated 
correspond  to  the  "wisdom"  or  experience  of  the  operator. 

The  data  resulting  from  such  a  game  of  operator  against  mission 
would  provide  a  most  useful  base  against  which  to  evaluate  the 
performance  of  actual  operators,  by  means  of  which  to  alter  training 
processes  and  materials,  and  so  on.  In  other  words,  if  we  can  use 
such  a  gaming  system  to  establish  the  best  procedure,  we  have 
obtained  an  absolute  standard,  with  all  its  attendant  advantages 
and  applications,  for  that  system  engaged  in  that  mission. 


A  CONCEPTUAL  SYSTEM 

There  must  be  ways  to  get  the  mission,  the  machine,  the 
operator,  the  man-machine  interaction  into  the  system. 

All  four  parts  must  be  capable  of  change.  A  new  machine 
configuration,  new  data  about  the  operator,  new  constraints  on  the 
mission,  new  models  of  man-machien  interaction,  all  must  be  put 
into  the  system  easily  without  interfering  with  what  is  not  changed 
and  with  the  basic  logic  of  the  system. 

This  means  that  there  must  be  a  superordinate  operating 
system  which  manages  the  various  parts  and  uses  information  from  the 
various  parts  to  accomplish  its  calculations. 


The  superordinate  system  must  have  the  following  characteristics 

It  can  call  upon  the  mission  description  to  find  out 
what  events  have  been  initiated  over  a  time  period 
of  length  t  and  what  events  will  be  initiated  up  to 
some  time  T,  in  the  future. 

It  can  call  on  the  machine  system  data  to  determine 
whether  the  machine  can  follow  the  trajectory  (in 
multi-dimensional  space)  required  by  the  mission 
events  in  time  t+T. 

It  can  call  on  the  man-machine  interaction  data  to 
determine  what  the  operator  will  be  required  to  do 
to  drive  the  machine  along  the  trajectory  demanded 
by  the  mission. 

It  can  call  on  the  operator  data  to  determine  whether 
the  HO  can  do  what  is  needed,  and  how  much  of  HO’s 
capacity  is  required  to  do  it. 

It  will  then,  if  all  the  foregoing  do  not  lead  to  a 
negative  conclusion,  play  the  game  to  establish  the 
optimum  path  in  the  space  of  time  and  behaviours  for 
the  HO  to  follow.  The  process  will  be  repeated  at 
each  time  instant,  t,  throughout  the  mission. 
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Since  there  are  three  components  involved  in  the  calculation 
of  the  load  imposed  on  the  HO  by  the  mission,  and  assuming  that  the 
descriptive  process  has  been  done  for  the  mission  and  the  system, 
then,  even  without  a  thorough  knowledge  of  HO*s  capacities  and  of 
the  man-machine  interactions,  it  would  be  possible  to  compute, 
conceptually,  the  minimum  characteristics  that  MO  would  have  to  have 
to  meet  the  demands  of  that  mission  with  that  system.  Similarly, 
if  one  knew  HO  and  the  system,  one  could  establish  the  boundary 
conditions  within  which  the  mission  would  have  to  fall  to  be  feasible. 
And,  of  course,  given  a  necessary  mission  and  the  capabilities  of 
the  HO,  one  could,  conceptually  at  least,  calculate  the  boundary 
conditions  within  which  the  system  would  have  to  fall  to  be  useable. 
Given  any  two^  one  can  determine  the  limits  for  the  third  which 
would  still  permit  successful  accomplishment. 

Since  missions  are  what  is  wanted,  it  is  unlikely  that  in  the 
first  instance  one  would  attempt  to  calculate  the  limiting  missions 
unless  there  were  severe  limitations  on  the  kind  of  hardware  and 
human  beings  one  could  imagine  applying  to  the  mission.  Therefore 
we  would  assume  that  mission  descriptions  would  be  a  given  input 
to  the  estimation  process.  Then,  given  HO  capabilities,  one  could 
gain  some  insight  into  the  directions  of  new  equipment  development 
which  would  be  needed  to  do  the  missions;  or  for  a  given  system, 
one  could  determine  either  whether  it  could  do  the  mission  with  the 
available  personnel  or  what  additional  information  about  HO  one 
would  need  to  get  to  answer  the  question  of  feasibility. 
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CONCLUSION  AND  RECOMMENDATIONS 

There  appears  to  be  no  simple  reductionist  solution  to  the 
problem  of  estimating  the  workload  imposed  on  a  human  operator  by 
a  system  engaged  in  a  mission  under  human  control  and  supervision. 

This  is  the  direct  consequence  of  the  fact  that  the  elements  of  a 
complex  task  are  not  immutably  fixed  in  time  and  sequence.  Thus 
most,  if  not  all,  elements  may  be  moved  about  in  time  and  reversed 
or  otherwise  changed  in  sequence  without  altering  the  success  or 
efficiency  of  the  system.  From  the  point  of  view  of  the  Human 
Operator  (HO) ,  there  is  much  more  flexibility  in  scheduling  the 
allocation  of  resources  and  this  is  particularly  helpful  when  HD's 
resources  are  strained  by  the  demands. 

As  we  have  indicated  earlier,  one  of  the  major  problems  is 
that  of  mission  description.  Historically,  missions  have  been 
analyzed  into  successive  units.  By  using  sufficiently  small  elements 
as  units,  reference  could  be  made  to  an  HO  data  file.  In  many  cases 
simple  experiments  had  already  been  performed,  often  by  laboratory 
psychologists — so  that  information  was  available  on  mean  required 
times,  variabilities  in  populations,  and  error  rates.  In  other  cases 
recourse  could  be  taken  to  simple  ad  hoc  experiments.  This  is  the 
method  of  mission  analysis.  What  is  being  proposed  here  is  its 
eventual  replacement  by  mission  characterization.  First  we  shall  list 
some  shortcomings  of  mission  analysis.  Second  wo  shall  describe  the 
steps  that  are  required  for  the  development  of  mission  characterization. 


-  39  - 


It  may  be  doubted  that  it  is  reasonable  to  expect  a  good  fit 
between  the  elemental  data  provided  thus  far  by  the  laboratory  and 
skilled  behaviour  in  the  real  world.  Following  are  some  of  the 
concerns : 


1.  Laboratory  conditions  favor  fast  responses  at  the 
cost  of  a  fairly  high  error  rate.  However,  it  is  typical 
that  the  sub-tasks  of  missions  must  be  performed  with 
practically  no  errors.  While  a  5%  error  rate  is  considered 
low  in  a  reaction-time  experiment  it  is  intolerable  in  the 
control  of  a  high-speed  vehicle. 

2.  Laboratory  experiments  are  performed  on  tasks  on 
which  the  subject  usually  has  had  very  little  practice.  We 
now  have  two  reasons  for  suspecting  the  generalities  provided 
by  laboratory  research.  We  do  not  know,  for  example,  whether 
Hick's  Law  of  the  logarithmic  relation  between  uncertainty 
and  latency  is  applicable  to  skilled  performance.  First, 
reaction-time  performance  has  usually  been  characterized  by 
high  error  rates.  Second,  data  have  been  obtained  from 
unskilled  persons. 

3.  With  the  exception  of  tracking,  single-episode 
tasks  have  prevailed  in  the  laboratory.  One  cannot  infer 
from  the  time  of  a  single  stimulus-response  the  amount  of 
time  that  will  be  required  if  that  response  is  embedded  in 
serial  activity.  The  speed  of  a  pianist's  finger  movements 
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is  far  faster  than  would  be  provided  by  estimates  from 
reaction  time. 

4.  Where  control  may  be  described  as  supervisory,  it 
is  already  understood  that  hierarchical  organization  exists, 
which  in  turn  implies  intelligent  behaviour.  HO  is  allowed 

to  move  time  windows,  within  limits,  to  select  among  operations, 
adding  some,  deleting  others,  etc.  Stimulus-response  elements, 
which  are  the  units  of  analysis,  cease  to  dominate  behaviour. 

5.  However,  it  is  likely  under  time  stress  that 
behaviour  will  revert  to  being  less  intelligent  parallelling 
Bartlett's  findings  on  the  effects  of  fatigue.  Thus,  even 

a  more  sophisticated  analysis  may  not  be  successful  since 
there  is  no  way  of  dealing  with  the  essential  factor  of 
concern:  how  performance  changes  as  time  stress  increases. 

What  is  being  suggested  is  that  complex  tasks  be  characterized 
rather  than  analyzed  into  sequential  units  and  then  attempt  to  learn 
how  values  of  the  characterization  variables  influence  the  effect 
of  time  stress  on  performance.  Characterization  variables  might 
include  the  amount  and  difficulty  of  inspecting,  stabilizing, 
estimating,  etc.  Also,  a  mission  might  be  characterized  by  the 
stochastic  distribution  of  the  transactions.  Finally,  the  frequency 
of  possible  intelligent  actions  would  be  described.  We  hardly  have 
a  vocabulary  for  such  characterization,  but  we  can  expect  one  to 
emerge  from  a  program  of  research  that  may  have  the  following 


stepwise  structure. 
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1.  Observe  performance  in  real  tasks,  using  skilled 
operators,  and  devise  a  table  of  characteristics  for  each. 
Subject  the  operators  to  varying  degrees  of  time  pressure  in 
performing  a  mission.  Obtain  performance  measures  of  errors 
(graded  amount,  incidence  of  commission  and  omission  errors, 
etc.),  of  indications  of  intelligent  behaviour,  of  workload 
ratings. 

With  various  tasks  studied  in  this  way,  task 
characteristics  could  be  related  to  time  stress.  For  example, 
more  mechanical  tasks  might  show  a  gradual  decline,  but  tasks 
where  intelligence  operations  is  possible  might  show  more 
sudden  breakdowns.  It  is  also  possible  that  quite  different 
slopes  will  be  obtained  with  the  different  degrees  of  stochastic 
dependence,  other  variables  being  constant.  For  example,  it 
has  been  shown  that  the  requirement  to  change  set  more 
frequently  results  in  poorer  performance. 

2.  Devise  synthetic  tasks,  including  clear  indicators 
(usually  lacking  in  real  tasks)  of  when  intelligent  behaviour 
is  being  engaged  in.  Train  operators  in  these  synthetic  skills 
and  test  them  with  varying  degrees  of  time  stress.  This  would 
be  a  way  of  verifying  or  falsifying  hypotheses  derived  from 
the  real  tasks  in  (1)  above. 
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3.  Establish  the  validity  of  the  derived  characteristics 
by  devising  new  tasks  with  the  same  formal  characteristics 

but  with  considerably  different  concrete  transactions. 

4.  Using  characteristics  that  have  proved  to  be  rugged, 
develop  a  method  of  computer  simulation  of  HO  interacting 
with  a  device  on  a  mis. ion.  If  it  is  found  that  behaviour 
changes  in  type  with  time  stress  this  must  be  included  in  the 
simulation. 

5.  Characterize  the  human  operations  on  a  device  and 
mission  of  interest.  Subject  the  synthetic  operator  to 
variations  in  time  stress  by  computer  simulation  to  establish 
boundary  conditions  for  satisfactory  performance. 


SUMMARY 


Any  workload  estimation  technique  as  opposed  to  virtually  all 
workload  measurement  techniques  requires  a  comprehensive  description 
of  the  human  operator,  the  system  HO  is  to  use  and  the  mission  the 
man-machine  system  is  to  accomplish,  in  the  absence  of  any  one  of 
these  three  parts,  it  is  manifestly  impossible  to  make  a  rational 
estimate  of  the  load  imposed  by  that  system  doing  that  mission  on  an 
HO.  Simple  linear  description  of  the  sequence  of  events  that  will 
take  place  during  the  mission  is  not  enough.  Events  are  not  rigidly 
fixed  in  time  so  that  the  HO  has  significant  freedom  to  move  component 
tasks  around  when  things  pile  up.  In  a  complex  task,  the  number  of 
events  which  may  be  moved  can  be  large  with  resulting  great  difficultie 
for  the  analyst. 

Task  pacing,  self-  versus  machine-paced-,  is  discussed  and 
redefined  as  measures  on  continuous  scales  of  the  degree  to  which 
events  during  a  mission  are  time  and  sequence  bound.  Problems  with 
some  of  the  existing  techniques  are  discussed  and  some  possible  paths 
toward  solutions  are  suggested.  It  has  become  clear  that  the  present 
reductionist  methods  are  not  able  to  provide  satisfactory  models  of 
intelligent  human  operator  behaviour  which  allow  estimates  of  mental 
workload  to  be  obtained  from  computer-based  analysis  methods.  This 
leads  the  analyst  inevitably  toward  artificial  intelligence  as  the 
ultimate  solution  of  the  workload  estimation  problem.  Virtually 
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all  workers  in  the  field  agree  that  a  highly  trained  and  properly 
motivated  human  operator  can  provide  excellent  reliable  and  valid 
reports  on  the  subjectively  assessed  effort  required  to  perform. 

The  Cooper  rating  scale  is  an  example. 

Since  the  principal  validation  of  any  workload  estimation 
method  is  either  long  term  statistical  data  on  system  effectiveness 
and  performance  or  the  report  of  the  human  operators  of  the  system; 
and  since  the  former  is  usually  very  difficult  to  obtain,  one  is 
left  with  the  notion  that  the  simulation  of  an  intelligent  and 
introspective  participant-observer  coupled  with  an  adequate  system 
characterization  is  the  best  way  to  obtain  reliable  and  valid 


workload  estimates. 
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APPENDIX 

Intelligent  Behaviour  and  Time  Stress 

It  may  be  hypothesized  that  as  time  stress  (Siegel  £  Wolf,  19&9)  increases, 
there  will  come  a  point  at  which  the  employment  of  intelligent  behaviour  begins 
to  diminish.  If  this  hypothesis  is  true,  any  computer  simulation  that  includes 
intelligent  behaviour  by  HO  must  also  include  an  interaction  between  the  employment 
of  intelligent  behaviour  and  time  stress.  As  HO  becomes  skilled  on  a  task,  he 
learns  many  ways  of  making  the  task  easier  in  addition  to  the  acquisition  of  rote 
habits.  These  include  the  learning  of  permissible  temporal  limits  for  responses 
and  the  making  of  optimal  placements  of  responses  within  these  windows.  It  also 
includes  the  recognition  of  familiar  stimulus  patterns  so  that  a  standard  series 
of  responses  may  be  used.  Without  these  accomplishments  HO  is  in  danger  of 
encountering  overloads  from  time  to  time.  An  operator  who  employs  intelligent 
behaviour  may  perform  at  a  higher  rate  of  pacing  than  one  who  does  not.  However, 
it  should  be  evident  that  the  employment  of  intelligent  behaviour  itself  demands 
a  portion  of  HO's  limited  capacity.  It  is  here  suggested  that  as  rate  of  pacing 
increases  so  that  HO  is  unable  to  both  respond  and  plan,  the  planning  will  drop 
out  as  urgent  responses  are  made.  Since  intelligent  behaviour  is  necessary  to 
maintain  a  high  pace  there  is  necessarily  a  sharp  drop  in  the  quality  of  performance. 

A  parallel  may  be  found  in  the  effect  of  fatigue,  which  increases  time  stress 
by  reducing  capacity  rather  than  by  increasing  rate  of  objective  pacing.  In  the 
historical  study  by  Bartlett,  "Fatigue  following  highly  skilled  work"  (1943)  it 
was,  in  fact,  found  that  intelligent  behaviour  was  reduced,  as  is  evident  from  the 
following  statement:  "When  an  operator  was  fresh  a  glance  at  the  dominant  signals 
meant  an  interpretation  of  the  whole  panel,  and  a  movement  of  a  controlling  lever 
meant  something  that  the  machine  was  doing,  or  would  very  soon  begin  to  do.  As 
the  task  continued  the  panel  split  up,  so  that  it  became  twenty  or  so  separate 
recording  instruments.  And  the  controlling  movements  split  up  also,  so  that  when 


any  one  was  made  it  was  not  pictured  in  a  pattern  of  machine  control,  but  only 
as  the  correction  of  a  particular  instrument  reading."  Thus,  there  was  a  regression 
from  the  intelligent  control  of  an  integrated  system  to  mechanical  reaction  to 
the  separate  elements. 

An  initial  attempt  was  made  to  devise  a  test  that  would  illustrate  the 
reduction  of  intelligent  behaviour  as  time  stress  was  increased.  This  is  the 
Pointers  Test  that  is  attached.  The  subject  moves  from  left  to  right  in  judging 
the  relation  between  the  pointer  positions  on  a  three-dial  frame  with  the  frame 
to  its  left.  Basically  the  operator  has  to  decide  which  of  the  three  pointers 
moved  differently  than  the  other  two,  or  if  that  was  the  case.  The  instructions 
also  allow  the  subject  to  examine  earlier  parts  of  the  test  which  may  be  transferred 
directly  to  the  new  problems.  Six  college  students  were  tested  (one  of  whom, 
unfortunately,  did  not  understand  the  instructions)  with  time  allowance  in  voiced 
pacing  ranging  from  0.59  to  1.82  of  each  subject's  personal  required  time  (without 
the  option  of  intelligent  behaviour).  Only  the  subject  given  the  slowest  pacing 
made  consistent  use  of  the  opportunities  to  use  earlier  parts  of  the  test.  The 
two  subjects  who  were  paced  considerably  behond  their  personal  level  made  no  use 
of  that  option.  Interestingly,  the  subject  who  was  given  the  slowest  pacing 
tied  for  second  in  terminal  rate  of  performance  (relative  to  personal  rate). 

These  results  are  very  preliminary  and  only  suggestive.  The  method  of  voiced 
pacing  is  inadequate  for  putting  sufficient  speed  stress  on  the  subject  so  that 
performance  will  suffer.  Still  It  was  seen  that  removal  of  time  stress  may 
paradoxically  result  in  fast  performance  if  the  opportunity  exists  for  intelligent 
behaviour. 

The  test,  of  course,  is  severely  limited  in  the  kinds  of  intelligent 
behaviour  that  are  possible.  Especially  neglected  Is  HO's  ability  to  use 
flexible  timing  to  prevent  responses  from  piling  up.  To  study  this  aspect  of 


intelligent  behaviour  would  require  testing  with  a  more  direct  kind  of  pacing, 
which  is  not  possible  with  a  paper-and-penci 1  test.  Also,  the  test  contains 
very  few  items  so  skilled  performance  was  not  actually  obtained.  With  the  use 
of  more  direct  pacing,  opportunity  for  a  variety  of  intelligent  behaviours,  and 
highly  practiced  subjects  data  could  be  obtained  that  would  be  instructive  on 
the  reduction  of  intelligent  behaviour  and  the  breakdown  of  performance  that 
occurs  with  increase  of  time  stress.  Following  Sanders'  suggestion  of  assessing 
workload  by  "testing  the  task  to  its  1 imi t"  (1979)  a  simulated  mission  might  be 
evaluated  by  simply  plotting  performance  against  rate  of  pacing.  A  task  near 
the  margin  could  be  identified  as  such  because  very  little  increase  in  speed  of 
pacing  would  bring  about  a  breakdown  of  performance.  It  should  follow  that  if 
a  task  depends  very  strongly  on  intelligent  behaviour,  the  breakdown  in  performance 
will  occur  rather  suddenly  as  some  critical  rate  of  pacing  is  reached. 
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THE  POINTERS  TEST 


The  small  circles,  such  as  you  see  on  this  page  represent  dials.  The  short  line  on  each  circle 
represents  a  pointer  on  the  dial.  A  pointer  can  have  one  of  eight  positions,  thus: 

O0GQO0OO 

In  going  from  left  to  right  in  the  series  above,  the  pointer  has  rotated  clockwise,  one  step  at  a  time. 
However,  in  the  test,  a  pointer  may  rotate  in  the  opposite  direction,  counterclockwise ■  It  may  also 
move  more  than  one  step  from  one  frame  to  the  next,  or  not  move  at  all. 

Each  frame  of  the  test  consists  of  a  vertical  column  of  the  three  dials.  A,  B,  and  C.  In  the  example 
below  there  are  16  frames.  The  task  is  to  indicate  for  each  frame  whether  the  three  pointers  have 
rotated  in  the  same  way  from  their  positions  on  the  frame  to  the  left. 

Here  is  a  series  for  which  the  first  six  frames  have  already  been  marked. 
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You  are  to  continue  the  series  using  the  following  rules: 

1.  If  all  three  pointers  rotated  in  the  same  direction  (clockwise  or 
counterclockwise)  and  the  same  number  of  steps,  draw  a  bar  (— ) 
above  the  frame  number. 

2.  If  two  pointers  rotated  in  the  same  direction  and  the  same  number 
of  steps,  but  one  pointer  differed  either  in  direction  or  number  of 
steps,  write  down  the  letter  for  that  dial  (A,  B,  or  C). 

3.  If  all  three  pointers  rotated  differently  as  to  direction  or  number 
of  steps,  write  down  an  X. 


As  you  move  through  the  series  you  can  simplify  your  task  by  writing  in  the  trial  numbers  of 
previous  frames  instead  of  the  — ,  A,  B  C,  X  marks.  This  can  bo  done  when  the  prevous  marks 
are  correct  on  the  new  frames.  The  brackets  above  frames  11-12  tell  you  that  the  correct  marks 
occurred  in  the  range  4-7.  Thus,  you  could  have  written  in  5  and  6  instead  of  —  and  A.  Frames 8- 10 
are  identical  with  frames  1-3.  Thus,  you  could  have  written  in  1,  2,  and  3  instead  of  C,  — .  and  X. 
/You  may  finally  note  that  the  correct  entries  for  frames  13  16  arc  exactly  the  reverse  of  those  for 
frames  1-4.  Thus,  you  could  have  written  4,  3,  2,  and  1  instead  of  B,  B,  C,  and  X. 
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THE  POINTERS  TEST 


SCORING  KEY,  PAGE  1 


I  he  small  circles,  such  as  you  see  on  this  page  represent  dial...  The  shot  I  line  on  each  circle 
represents  a  pointer  on  the  dial.  A  pointer  can  have  one  o(  eight  positions,  thus: 

©0GOO0OO 

In  going  from  left  to  right  in  the  series  above,  the  pointer  has  rotated  clockwise,  one  step  at  a  time. 
However,  in  the  test,  a  pointer  may  rotate  in  the  opposite  direction,  counterclockwise  It  may  also 
move  more  than  one  step  from  one  frame  to  the  next,  or  not  move  at  all. 

Each  frame  of  the  test  consists  of  a  vertical  column  of  the  three  dials.  A,  B,  and  C.  In  the  example 
below  there  are  16  frames.  The  task  is  to  indicate  for  each  frame  whether  the  three  pointers  have 
rotated  in  the  same  way  from  their  positions  on  the  frame  to  the  left. 

Here  is  a  series  for  which  the  first  six  frames  have  atroady  been  marked. 


A© 

B© 

C© 


You  are  to  continue  the  series  using  the  following  rules: 

1.  If  all  three  pointers  rotated  in  the  same  direction  (clockwise  or 
counterclockwise)  and  the  same  number  of  steps,  draw  a  bar  (--) 
above  the  frame  number. 

2.  If  two  pointers  rotated  in  the  same  direction  and  the  same  number 
of  steps,  but  one  pointer  differed  cither  in  direction  or  number  of 
steps,  write  down  the  letter  for  that  dial  (A,  B,  or  C). 

3.  If  all  three  pointers  rotated  differently  as  to  direction  or  number 
of  steps,  write  down  an  X. 
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As  you  move  through  the  series  you  can  simplify  your  task  by  writing  in  the  trial  numbers  of 
previous  frames  instead  of  the  -  ,  A,  B  C,  X  marks  This  can  be  done  when  the  previous  marks 
are  correct  on  the  new  frames.  The  brackets  above  frames  11-12  tell  yott  that  the  correct  marks 
occurred  in  the  range  4-  7.  Thus,  you  could  have  written  in  5  and  6  instead  of-  -  and  A.  EramesO-IO 
arc  identical  with  frames  1-3.  Thus,  you  could  have  written  in  1,  ?,  and  3  instead  of  C,  -  ,  and  X. 
You  may  finally  note  that  the  correct  entries  for  frames  13  16  are  exactly  the  reverse  ol  those  for 
frames  1-4.  Thus,  you  could  have  written  4,  3.  2,  and  1  instead  of  B,  U,  C,  and  X. 
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,  SCORING  KEY,  PAGE  2 _ (including  alternative  trial~number  response^) 
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